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ABSTRACT 



A study followed for a year or more the internal experiences 
of learning and change of 41 adult basic education/English as a second 
language learners enrolled in these 3 programs: community college, family 
literacy site, and workplace site. It focused on learners' meanings as the 
starting point for exploration and used a developmental lens to understand 
learners' program experiences in its approach to data collection and 
analysis. Protocols and measures administered before or near the start day of 
each program, during the middle months, and near the end of the program were 
focus groups; Experiences of Learning Interviews; Subject -Object Interview; 
Loevinger's Ego Development Sentence Completion Test; Role-Related Vignettes; 
Satisfaction with Life Scale; Perceived Efficacy Beliefs Scale; Locus of 
Control Scale; Self as Learner, Parent, or Worker Map; Reflecting on Changes 
in Self as Learner, Parent, or Worker Map; classroom observations; and 
Teacher Interview on Student Changes. Key theoretical and methodological 
lessons that focused on exploring development ally oriented questions were 
adapting measures in accordance with learnings related to logistical, 
language and cultural issues; learnings related to contextual issues; and 
learnings related to using a range of measures to understand learners ' 
meaning making and program experiences. Protocols and measures are appended. 
(Contains 45 references .) (YLB) 
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INTRODUCTION 

Beyond the acquisition of important language skills and increased content learning, what are the 
bigger internal meanings for adults participating in ABE/ESOL programs? How do the 
systematic ways in which adults are making meaning when they enter their program shape how 
they will best learn in programs and what they will most need from them? How can 
developmental theory and research methods increase understanding of how ABE/ESOL learners’ 
experience their programs? 

As adult developmental psychologists interested in adult education 1 , we carefully 
followed for a year or more the internal experiences of learning and change of 41 ABE/ESOL 
learners who were enrolled in three different U.S. programs (i.e., a community college, a family 
literacy site, and a workplace site). Each program was oriented to enhancing participants’ 
greater English language fluency, content knowledge, and effectiveness as students, parents, or 
workers. Our purpose was to better understand how: these adults experienced program learning; 
this learning transferred to their roles as parents, workers, or learners; they experienced program 
supports and challenges to their learning; this learning helped them change. Our research 
methods enabled us to listen carefully to participants’ experiences and attend to their meaning 
making so that we were able to trace their processes of learning and, in some cases, their 
transformation. In this paper discussion I will discuss our research methods, interpretative 
techniques, and the encountered challenges and strategies for attending to them from our 
longitudinal, mixed methods study that addressed these questions and was funded by the 
National Center for the Study of Adult Learning and Literacy (NCSALL). 

This is the first in-depth study that examines adults’ meaning-making of their own 
learning experiences in three ABE/ESOL programs. Prior studies employing Robert Kegan’s 
(1982, 1994) theory have been mostly composed of white, highly educated, middle class 
American adults who speak English as their first language. Our research (Drago-Severson, 

1 I acknowledge and thank all members of the Adult Development Research Team of NCSALL, and especially our 
pnncip investigator. Professor Robert Kegan, whose collective wisdom infused this research project and team 
monograph (Kegan et al., 2001a; Kegan, Broderick, Drago-Severson, Helsing, Popp, & Portnow, 2001b). 



Adult Development Researchers’ Reflections 



3 



Helsing, Kegan, Broderick, Portnow, & Popp, 2001a; Kegan, Broderick, Drago-Severson, 
Helsing, Popp, & Portnow, 2001b) extends the use of this framework by applying it to adults 
who are not economically privileged, mostly not native-born American, and mostly non-native 
English speakers. 

This paper addresses questions of theoretical and practical importance: How do we 
conduct research that relies heavily on language (in order to understand both the content of a 
learner’s thoughts and to assess the structure of thinking— two central aspects of our study) with 
samples of ESOL learners? How relevant might a developmental theory be (given prior studies 
with strikingly different populations) to understand ESOL learners’ meaning making? What 
methodological challenges might be encountered in exploring these kinds of developmentally 
oriented questions with ESOL learners and how might we adapt measures and develop strategies 
in order to better understand ESOL learners’ perspectives on their program experiences and their 
meaning making? Here, I will discuss our research methods to illuminate how we addressed 
these questions and worked out encountered challenges. 

To provide context for this methodological discussion, first I will briefly review the 
theoretical frameworks that guided our study and informed our approach to data collection and 
analysis. Next, I will present a brief description of each quantitative or qualitative measure. 

Each description is followed by a discussion of what we learned from administering each 
measure and how and why we adapted several standard measures, traditionally employed with 
populations who are native English speakers, to better understand the experiences of the 
ABE/ESOL sample we studied. Four themes are highlighted in discussing how we adapted and 
learned from administering various measures: (1) logistical issues, (2) language issues, (3) 
cultural issues, and (4) contextual issues. This paper illuminates what we learned about the 
benefits of employing multiple research methods — informed by various theoretical and 
methodological paradigms — given the visible and less visible challenges of research with this 
highly diverse population. In closing I emphasize how using a range of measures helped in 



Adult Development Researchers’ Reflections 



4 



triangulation of data and with assessing validity, and the importance and usefulness of this 
research. 

Beyond increasing understandings of how ABE/ESOL learners can be better supported in 
ABE/ESOL programs, the longer term objectives of this research are to improve teaching and 
learning practices, enhance program design by using research to inform practice, and deepen 
understandings about the value of using multiple research methods and various types of 
developmental measures to inform and broaden our understanding of adult learners’ experiences. 
This work will help us to attend mindfully to the qualitatively different ways in which learners 
make sense of their ABE experiences and it holds the potential to strengthen future research, 
classroom practice, and program design. I offer our study’s research methods and lessons 
learned as resources for other researchers, practitioners, developmentalists, and policymakers. 



CONTEXT AND THEORETICAL FRAMEWORK 2 

In the growing ABE/ESOL field, researchers cite the need for in-depth qualitative studies that 
are not framed from the perspective of either the ABE/ESOL mission, in general, or the purposes 
of the specific ABE/ESOL program in which the learner is enrolled (Skilton-Sylvester & Carlo, 
1998, Rockhill, 1982; Valentine, 1990; Horsman, 1990). Wiley (1993) urges researchers to focus 
on learners perspectives on their own experiences, hopes, and needs, instead of examining the 
learner s perspective in relationship to a program’s expectations or to the U.S. host society’s 
definitions of the learner’s needs. Furthermore, Taylor (1996) highlights the value of using 
developmental theory to inform our understanding of ABE learners’ experiences (Kegan et al., 
2001a; Kegan et al., 2001b). 

Our study responds to these calls in two ways. First we focus on learners’ meanings as 
the starting point for exploration. Secondly, in our approach to data collection and analysis, we 
employ a developmental lens to understand learners’ program experiences. In so doing we join 
Lytle and her colleagues (Lytle, 1991; Lytle & Schultz, 1990; Lytle, Marmor & Penner, 1986) in 
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their explicit call to help develop a literature of “adults’ own evolving conceptual frameworks or 
theories about language, literacy, teaching, and learning” (Lytle, 1991, p. 120). To this end we 
designed qualitative protocols and adapted standard quantitative measures that enabled us to 
thoughtfully and carefully attend to and examine both the meaning constitutive and potentially 
transformable nature of adult learners’ beliefs (Kegan et al., 2001b). In this work, we also 
utilized Robert Kegan ’s (1982, 1994) constructive developmental theory to understand how the 
adults in this study made sense of what they learned in their programs and the supports and 
challenges they named as facilitating their growth. This theoretical framework is informed by 30 
years of research in the adult development field, which suggests that developmental principles 
can be applied to adults (Basseches, 1984, Belenky et al., 1986; Cranton, 1994, 1996; Daloz, 
1986, 1999; Kegan 1982, 1994; Kohlberg, 1984; Mezirow, 1991; Piaget, 1965; Weathersby, 
1976). Our research methods enabled us to focus on how meaning systems shape and frame 
experience, constituting a lens through which the adult learner looks out at the world within and 
beyond the classroom, and how that lens can potentially change over time (Kegan et al., 2001a). 

Development, from our perspective, involves more than learning new skills or acquiring 
knowledge, which we refer to as informational learning. Development also rests upon 
transformational learning — learning that involves a qualitative shift in how people know and 
understand themselves, their worlds, and the relationship between the two. Transformational 
learning enables people to take broader perspectives on themselves (seeing and understanding 
different aspects of the self) and others (Cranton, 1994; Kegan, 1982, 1994; Mezirow, 1991, 
2000). Transformational change is intimately linked to the way people construe their adult 
responsibilities and roles. This kind of learning helps them enhance their internal capacities to 
better manage the complexities of their lives as learners, parents, and workers (Drago-Severson 
et al., 2001a; Drago-Severson, Helsing, Kegan, Broderick, Popp, & Portnow, 2001b; Helsing, 
Portnow, Popp, & Broderick, 2001; Kegan et al., 2001a; Kegan et al., 2001b). 



Portions of this section appear in similar form in our monograph (Kegan et al., 2001b), and/or its executive 
summary (Kegan et al., 2001a), and/or articles written for practitioners (Drago-Severson et al., 2001a). 
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Kegan’s theory enabled us to consider the ways in which people construct the reality in 
which they live and the ways in which these constructions can change or develop over time. We 
refer to an adult’s underlying meaning system — through which all experience is filtered and 
understood — as a way of knowing or a developmental level 3 or a developmental level (Kegan et 
al., 2001b). A person’s way of knowing organizes how she understands her experience of 
herself, others, and life events and situations. Each way of knowing has its own logic, which is 
different from and builds upon the previous logic by incorporating the former into its new 
meaning system. Moving from one developmental level to another is a progression of increasing 
complexity in an individual’s cognitive, emotional, interpersonal, and intrapersonal capacities. 
Understanding how a person is making sense of her world creates an opportunity to join her and 
offer support in a way that she will experience as being supportive (Drago- Severson et al., 
2001a). Three qualitatively different ways of knowing are most prevalent in adulthood: the 
Instrumental, the Socializing, and the Self-Authoring ways of knowing. 

A person’s way of knowing shapes how he understands his responsibilities as a learner, 
parent, family member, and worker, and how he thinks about what makes a good teacher, a good 
student, a good parent, and a good employee. We employed this lens to inform our research 
design, protocols, and analysis. It influenced the framing and nature of our questions, which 
were aimed at understanding how individual participants made sense of their motives and goals 
for learning, their expectations for themselves as learners and for their teachers, the supports and 
challenges to their learning, and their sense of themselves in their social roles. This theoretical 
framework also allowed us to trace how participants’ sense making changed — grew more 
complex — over time. In the next section I will discuss our research methods and give special 



In this writing the terms way of knowing and developmental level are used interchangeably. Belenky et al.’s 
important work, especially Women’s Ways' of Knowing (1986), has achieved such prominence in the adult education field, that it 
is important to stress that we use the term “way of knowing” in its literal and ordinary sense here; we are not referring to their 
taxonomy. In our study, we employ the term way of knowing to refer to the underlying structure of a person’s meaning making: 
the subject- object relationship — a person’s developmental level. “Subject” refers to what a person with a particular way of 
knowing is embedded in, identified with, cannot reflect upon, and cannot take a perspective on. “Object” refers to what a person 
with a particular way of knowing can examine, and have perspective on. The distinctly different meaning-systems defined in our 
study are qualitatively different ways of organizing the subject-object relationship, and thus, experience. They are literally 
different “ways of knowing” and understanding reality (Kegan et al., 2001a; Kegan et al,, 2001b). 
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attention to our research protocols and learnings from their administration in service to how they 
helped us explore our research questions. 



METHODS 4 

The following research questions guided our exploration of the developmental dimensions of 
transformational learning: 



• How does developmental level shape adults’ experiences and definitions of the core roles 
they take on as learners, parents, and workers? 

What are the regularities in the ways in which adults at similar levels of development 
construct the role demands and supports in each of these domains? 

• How do adult learners’ ways of knowing shape their experience and definition of programs 
dedicated to increasing their role competence? 

What are adult learners’ motives for learning, definitions of success, conceptions of the 
learners’ role, and understandings of their teachers’ relationship to their learning? 

• What educational practices and processes contribute to changes in the learner’s relationship 
to learning (vis-a-vis motive, efficacy, and meaning system) and specifically to any re- 
conceptualizations of core roles? 



Site Selection: 

The sites we chose were ongoing programs that we considered to model best practice (see 
e.g., Harbison & Kegan, 1999). Best practice programs use effective methods for achieving 
excellent, targeted results, and they set benchmarks for other programs to follow (Hammer & 
Champy, 1993). We selected three programs in Massachusetts that were longer term (nine to 14 
months), enabling us to explore long-term growth in students’ understanding and allowing us to 
examine the developmental dimensions of transformational learning (Drago-Severson et al., 
2001a; Drago-Severson & Berger, 2001; Kegan et al., 2001a). 

The selected programs incorporated a variety of supports and challenges to facilitate adult 
learning (e.g., tutoring, advising, and technological support for learners). The goal of the 
programs was either to (a) prepare learners for enrollment in a General Education Diploma 

4 Portions of this section appear in similar form in Chapter Two, Research Methods, of our monograph (Kegan et 
al., 2001b) and in an article written for a practitioner audience (Drago-Severson et al., 2001b). 
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(GED) program or to help students learn English (ESOL); or (b) to prepare learners for academic 
coursework in college; or (c) to earn a high school diploma. These three programs also had 
developed curricula aimed at improving adults’ specific role competency in one of three social 
roles, student, parent, or worker. We designed protocols to help us examine and trace how 
participants, over time, reported program learning as helping them perform specific social roles 
differently. Therefore, each individual measure that was administered at a particular site (e.g., 
the participant interview #1, which was administered before or near program entry to 
participants, contained similar questions at all three sites and also include questions related to a 
particular social role). We also developed protocol questions to explore how program design, 
teacher practice, and curricula might support and challenge learners with different ways of 
knowing and possibly lead to transformation. 

Selection of Participants- 

During 1998-99, we followed 41 adult learners originating from different regions in the 
world who were enrolled in the three selected programs. This sample was diverse with respect to 
race, ethnicity, age, past educational experiences, socioeconomic status, and social roles. The 
vast majority of participants across these three sites were non-native English speakers. 

At Bunker Hill Community College (BHCC), Charlestown, MA, we studied how a group 
of newly immigrated young adults (late teens or early 20s) experienced a pilot program aimed at 
helping them become better prepared for academic coursework in college. These adult students 
were enrolled in the same two BHCC classes during their first term (i.e., an ESOL class, and also 
an introductory psychology class designed for ESOL learners). During the second semester the 
group separated, and each student independently selected his or her own courses from the range 
of academic courses at BHCC. Similar to adults at our other two sites, all students enrolled in 
this program were primarily from lower socioeconomic backgrounds and spoke English as a 
Second Language. Unlike learners at our other two sites, these students had already earned a 
high school diploma and were matriculating for an Associate degree or a Certificate of Study. 
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Our interest was in learning how participation in this program influenced the ways in which 
participants conceived their roles as students. 

At the second site, we followed two groups of parents enrolled in a Family Literacy 
Program s . One group was in a pre-GED class and the second parent group was enrolled in an 
ESOL class. These parents (most in their 30s) emigrated from different countries and had been 
living in the United States for an average of nine years. Parents in this program also had at least 
one child who attended the Family Literacy Program. Our interest was in learning how 
participation in this family literacy program affected the ways in which these adults conceived 
and enacted their roles as parents. 

At the Polaroid Corporation manufacturing plant in Norwood, MA, our third site, we 
studied a group of workers who participated in a 14-month CE1 Adult Diploma Program that was 
designed and delivered by the Continuing Education Institute (CEI) of Watertown, MA. Most of 
these learners were in their 30s and 40s, had lived in the U.S. for more than 20 years, were 
married, and had children. Our interest was in learning how participation in this program 
affected the ways in which these individuals conceived and enacted their role as workers. 

All adults enrolled in these programs were invited to participate in our study. At each 
site, all participants initially agreed to participate in our research. We began with fifty-eight 
participants (17 from BHCC, 22 from the Family Literacy site, and 19 from Polaroid); however, 
during the research seventeen participants (across sites) either withdrew or temporarily stopped- 
out of their programs for a variety of reasons (Drago-Severson et al., 2001a; Kegan et al., 

2001a). We conducted non-completer interviews with a few of these participants after the 
programs’ year ended. 

Data Collection: 

Forty-one adults participated in the complete study, making time available on three (and, at the 
one site, four) separate extended occasions to share their thinking via a variety of data collection 
methods and tools. These included tape-recorded, open-ended qualitative interviews; structured 



Adult Development Researchers’ Reflections 10 



exercises; classroom observations; focus groups; and survey type measures. Table 1 presents our 
data collection schedule. 



Table 1 : Schedule of Data Collection at Research Sites 



SITE 


ROUND ONE 


ROUND TWO 


ROUND THREE 


ROUND FOUR 


Community 

College 


October 1998 

(several hours on 
two separate days) 


December 1998 

(several hours on 
two separate days) 


May 1999 

(several hours on 
two separate days) 




Family 

Literacy 


November 1998 

(several hours on 
two separate days) 


March 1999 

(several hours on 
two separate days) 


July 1999 

(several hours on 
two separate days) 




Workplace 


March/April 1998 

(several hours on 
two separate days) 


September 1998 

(several hours on 
two separate days) 


March 1999 

(several hours on 
two separate days) 


June 1999 

(several hours on 
two separate days) 



(Drago-Severson, 2001) 



As Table 1 indicates, three (and at one site four) intensive rounds of data collection 
occurred over the course of a year or more. Additionally, at all sites, we conducted classroom 
observations during the academic year, program teacher interviews at the start and toward the 
end of the academic year, and program director interviews at the start and, in some cases, the end 
of the year. In total, we conducted and analyzed approximately 670 hours of semi-structured 
qualitative interviews and developmental assessments (tape-recorded and transcribed), 160 hours 
of quantitative survey type measures, 25 hours of observations, and various documents. 

Although we considered interviewing each adult learner in his or her first language, the 
first-language diversity of our sample made the cost of this strategy prohibitive and impractical. 
All interviews were administered individually, in English (discussed later in more detail). Each 
site visit lasted several hours on at least two different days during each round of data collection 
allowing us to gather data on a wealth of questions about participants’ experience of a variety of 
aspects of the learning and teaching enterprise. For example, questions included: What are your 
purposes in pursuing this learning? What, in your view, makes a person a good teacher? What 
effect is your learning having on your work, in your relationships with your child, or in your role 
as a prospective college student? These and other questions helped us examine what the 



5 This site prefers to remain anonymous. 
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processes of transformational learning looked like, how learners with different developmental 
levels experienced such processes, and the practices that learners named as supportive to these 
changes. Revisiting the same participant at different points allowed us to ask him about any 
changes he noticed in himself and to trace these changes by examining data from different points 
in time. 



Qualitative and Quantitative Measures: Rounds of Administration 

Following is an overview of the qualitative interviews, structured exercises, and survey type 
quantitative measures that we administered to participants at all three research sites. In addition 
to describing the measures and their intended purposes, I focus on how, if at all, we adapted them 
(and why we did so) during the course of this research. Specifically, I highlight how our team’s 
process of reflecting on what was working well and what needed to be improved upon— before, 
during, and after each round of data collection — helped us, in many cases, to improve the 
measures and our administration of them to this diverse ABE/ESOL sample. Revisiting and 
revising our protocols in this way assisted us in better-understanding participants’ meaning 
making. Table 2 displays the types of measures we administered to learners and when we 
administered them to participants at all sites. Note that at one site, the workplace site, we 
administered two middle rounds of measures, since this the CEI Adult Diploma Program was 14 
months in duration. 

Four central themes 6 emerge from our learnings related to our administration of various 
protocols. These are woven through and highlighted in discussing each measure, and revisited in 
the final section of this paper. 

Logistical Issues: For example, (A) Time constraints for data collection made it necessary 
to alter and shorten certain protocols so that they could be administered to learners at each 
site. B) Pairing the same interviewer and interviewee, whenever possible, helped to build 
research relationships. (C) Gaining access to participants who did not complete of the 
three programs was sometimes difficult. 
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